UNCLASSIFIED 


AD 

AD-E403  957 


Technical  Report  ARWSE-CR-1 6004 


TACTICAL  APPLICATIONS  (TACAPPS)  USER  DESIGN  WORKSHOP 
ANALYSIS  AND  FINDINGS  REPORT 


Aaron  Lieb 
Ross  Arnold 
U.S.  ARMYARDEC 


Joshua  Cauvel 
Lauren  Grim 
General  Dynamics,  Inc. 
Florham  Park,  NJ 


November  2017 


U.S.  ARMY  ARMAMENT  RESEARCH,  DEVELOPMENT  AND 
ENGINEERING  CENTER 

Always  a  Step  Ahead 


ARDEC 

MARMAMENTS I 


Weapons  and  Software  Engineering  Center 
Picatinny  Arsenal,  New  Jersey 


Approved  for  public  release;  distribution  is  unlimited. 


UNCLASSIFIED 


UNCLASSIFIED 


The  views,  opinions,  and/or  findings  contained  in  this  report  are  those  of  the 
author(s)  and  should  not  be  construed  as  an  official  Department  of  the  Army 
position,  policy,  or  decision,  unless  so  designated  by  other  documentation. 

The  citation  in  this  report  of  the  names  of  commercial  firms  or  commercially 
available  products  or  services  does  not  constitute  official  endorsement  by  or 
approval  of  the  U.S.  Government. 

Destroy  by  any  means  possible  to  prevent  disclosure  of  contents  or 
reconstruction  of  the  document.  Do  not  return  to  the  originator. 


Approved  for  public  release;  distribution  is  unlimited. 

UNCLASSIFIED 


UNCLASSIFIED 


REPORT  DOCUMENTATION  PAGE 


Form  Approved 


OMB  No.  0704-01-0188 


The  public  reporting  burden  for  this  collection  of  information  is  estimated  to  average  1  hour  per  response,  including  the  time  for  reviewing  instructions,  searching  existing  data  sources, 
gathering  and  maintaining  the  data  needed,  and  completing  and  reviewing  the  collection  of  information.  Send  comments  regarding  this  burden  estimate  or  any  other  aspect  of  this  collection 
of  information,  including  suggestions  for  reducing  the  burden  to  Department  of  Defense,  Washington  Headquarters  Services  Directorate  for  Information  Operations  and  Reports  (0704-0188), 
1215  Jefferson  Davis  Highway,  Suite  1204,  Arlington,  VA  22202-4302.  Respondents  should  be  aware  that  notwithstanding  any  other  provision  of  law,  no  person  shall  be  subject  to  any 
penalty  for  failing  to  comply  with  a  collection  of  information  if  it  does  not  display  a  currently  valid  OMB  control  number. 

PLEASE  DO  NOT  RETURN  YOUR  FORM  TO  THE  ABOVE  ADDRESS. 


1 .  REPORT  DATE  (DD-MM-YYYY)  2.  REPORT  TYPE  3.  DATES  COVERED  ( From  -  To) 

November  2017  Final  Nov.  2,  2015  through  Nov.  4,  2015 

4.  TITLE  AND  SUBTITLE 

TACTICAL  APPLICATIONS  (TACAPPS)  USER  DESIGN 
WORKSHOP,  ANALYSIS  AND  FINDINGS  REPORT 

5a.  CONTRACT  NUMBER 

5b.  GRANT  NUMBER 

5c.  PROGRAM  ELEMENT  NUMBER 

6.  AUTHORS 

Aaron  Lieb  and  Ross  Arnold  -  U.S.  Army  ARDEC 

Joshua  Cauvel  and  Lauren  Grim  -  General  Dynamics 

5d.  PROJECT  NUMBER 

5e.  TASK  NUMBER 

5f.  WORK  UNIT  NUMBER 

7.  PERFORMING  ORGANIZATION  NAME(S)  AND  ADDRESS(ES) 

General  Dynamics  U.S.  Army  ARDEC,  WSEC 

7-9  Vreeland  Rd.  Fire  Control  Systems  &  Technology  Directorate 

Florham  Park,  NJ  (RDAR-WSF-M) 

07932  Picatinny  Arsenal,  NJ  07805-5000 

8.  PERFORMING  ORGANIZATION 

REPORT  NUMBER 

9.  SPONSORING/MONITORING  AGENCY  NAME(S)  AND  ADDRESS(ES) 

U.S.  Army  ARDEC,  ESIC 

Knowledge  &  Process  Management  (RDAR-EIK) 

Picatinny  Arsenal,  NJ  07806-5000 

10.  SPONSOR/MONITOR’S  ACRONYM(S) 

11.  SPONSOR/MONITOR’S  REPORT 
NUMBER(S) 

Contractor  Report  ARWSE-CR-1 6004 

12.  DISTRIBUTION/AVAILABILITY  STATEMENT 


Approved  for  public  release;  distribution  is  unlimited. 


13.  SUPPLEMENTARY  NOTES 


14.  ABSTRACT 

As  part  of  the  Tactical  Applications  (TacApps)  effort  to  design  a  highly  user-friendly,  relevant,  and 
capable  software  system  for  the  common  operating  environment  (COE)  v3,  engineers  at  the  U.S.  Army 
Armament  Research,  Development  and  Engineering  Center,  Picatinny  Arsenal,  NJ,  Weapons  and  Software 
Engineering  Center,  devised  and  facilitated  a  two-day  design  workshop  at  1st  Armored  Division  headquarters 
in  Fort  Riley,  KS.  The  workshop  consisted  of  several  exercises  designed  to  uncover  the  natural  tendencies, 
viewpoints,  terminology,  and  workflows  of  target  users  performing  specific  tasks.  A  total  of  46  warfighters  were 
included  in  the  exercises).  The  TacApps  team  was  able  to  gather  significant  insight  through  virtue  of  the 
design  session.  This  insight  heavily  influenced  the  further  design  and  development  of  COE  v3  software  for 
Mission  Command  systems. 


15.  SUBJECT  TERMS 

Tactical  Applications  (TacApps)  Command  Post  Client  (CPC)  Command  Post  Design 
Common  operating  environment  (COE)  v3  Mounted  computing  environment  (MCE) 
Command  Post  Computing  Environment  (CPCE)  Workshop  User  jury  Software  design 
Human  factors 


16.  SECURITY  CLASSIFICATION  OF: 

17.  LIMITATION  OF 
ABSTRACT 

18.  NUMBER 
OF 

19a.  NAME  OF  RESPONSIBLE  PERSON 
Ross  Arnold 

a.  REPORT 

u 

b.  ABSTRACT 

u 

c.  THIS  PAGE 

u 

SAR 

PAGES 

51 

19b.  TELEPHONE  NUMBER  (Include  area 

code)  (973)  724-8618 

Standard  Form  298  (Rev.  8/98) 

Prescribed  by  ANSI  Std.  Z39.18 


UNCLASSIFIED 


UNCLASSIFIED 


CONTENTS 

Page 

Executive  Summary  1 

Bottom  Line  Up  Front  (BLUF)  1 

Sessions  1 

Topics  1 

Structure  2 

User  Feedback  2 

Next  Steps  3 

Detailed  Report  3 

Goals  3 

Topics  4 

Methodology  Overview  5 

Focused  Small  Groups  6 

Card  Sorting  6 

Limited  Exposure  6 

Participant  Demographics  7 

Participant  Ranks  7 

Mobile  Operating  System  (OS)  Favored  by  Participants  7 

Participant  Ages  8 

Participant  Education  Levels  9 

Participant  Command  Post  of  the  Future  (CPOF)  Expertise  9 

Participant  Technical  Expertise  10 

Workshop  Findings  10 

Session  1:  Language  of  Collaboration  10 

Session  2:  Roles  and  User  Identities  18 

Session  3:  Product  and  Workspace  Management  23 

Design  Affirmations  25 

Individual  View,  Shared/View  25 

Folder  Design  26 

Mounted  26 

Briefing  Concept  26 

Storing  Data  in  “My  Folder"  27 

Folder/Groups  Design  27 

Insisting  on  Simplicity  27 

Tooltips  for  Instruction  27 

Attaching  Chat/Email/other  Messages  to  Products  27 

Right-click  Menus  27 

Limited  Need  for  Three-dimensional  Map  Collaboration  28 

Fixed  Board  Layouts  28 


Approved  for  public  release;  distribution  is  unlimited. 

UNCLASSIFIED 


UNCLASSIFIED 


CONTENTS 

(continued) 

Page 

Ownership  Basics  28 

Design  Challenges  28 

Ownership  Details  28 

Personal  Folder  28 

Task  Organization  29 

Need  for  Authoritative  Products  29 

Collaboration  Ideas  29 

System  Interoperability  29 

User  Identity  30 

Fundamental  System  Types  30 

Product-centric  Organization  30 

Adding  to/Changing  Unowned  Products  30 

Workflow  Needs  30 

Next  Steps  31 

Conclusions:  Event  Lessons  Learned  32 

Survey  33 

Presentation  33 

Appendix  -  Backup  Data  35 

Distribution  List  43 

FIGURES 

1  TacApps  design  workshop  team  5 

2  Participants  and  the  design  team  at  1 1D  headquarters  6 

3  Diversity  of  ranks  within  the  division  and  brigade  sessions  7 

4  Mobile  OS  preferences  8 

5  Participant  age  range  8 

6  Participant  education  levels  9 

7  CPOF  range  of  expertise  9 

8  Technical  level  of  expertise  10 

9  Findings,  terminology  11 

Approved  for  public  release;  distribution  is  unlimited. 

UNCLASSIFIED 

ii 


UNCLASSIFIED 


FIGURES 

(continued) 

Page 

10  Findings,  data  presentation  needs  11 

11  Findings,  use  of  collaboration  12 

12  Map  checklist  13 

13  Compilation  of  map  usage  data  13 

14  Basic  understanding  of  the  usage  of  the  term  “collaboration”  within  CPOF  14 

15  Statements  that  describe  map  collaboration  video  15 

16  Questions  indicating  how  ownership  concepts  would  apply  in  a  collaborative  workflow  15 

17  Results  of  how  ownership  rules  apply  to  a  board  found  in  a  folder  16 

18  Terminology  preferences  16 

19  Naming  of  individual  versus  shared  views  17 

20  Most  important  initial  tools  17 

21  Onboarding  example  likes  and  dislikes  18 

22  Findings,  social  media  expectations  19 

23  Findings,  workflow  needs  20 

24  Most  useful  contacts  list  21 

25  Example  user  identities  22 

26  Role  attributes,  relative  popularity  -  exercise  results  22 

27  Findings,  information  organization  23 

28  Findings,  mounted/dismounted  needs  24 

29  TacApps  “groups”  menu  item  example  25 

30  Participants’  app  ideas  32 

Approved  for  public  release;  distribution  is  unlimited. 

UNCLASSIFIED 

iii 


UNCLASSIFIED 


ACKNOWLEDGMENTS 

The  authors  would  like  to  thank  Timothy  Rybarski  and  Gregory  Roehrich  (U.S.  Army 
Armament  Research,  Development  and  Engineering  Center,  Picatinny  Arsenal,  NJ)  for  their 
sponsorship  and  support,  as  well  as  the  Tactical  Mission  Command  Product  Management  Office  for 
funding  the  Weapons  and  Software  Engineering  Center  to  undertake  this  effort. 


Approved  for  public  release;  distribution  is  unlimited. 

UNCLASSIFIED 


v 


UNCLASSIFIED 


EXECUTIVE  SUMMARY 


Bottom  Line  Up  Front 

The  Tactical  Applications  (TacApps)  design  team  held  a  two-day  design  workshop  with 
Soldiers  at  Fort  Riley,  KS,  on  November  2  and  3,  2015. 

The  goal  of  the  workshop  was  to  elicit  user  feedback  to  help  guide  and  refine  the  design  of 
the  TacApps  system.  With  a  total  of  46  participants  (made  up  of  40  unique  participants,  as  some 
Soldiers  stayed  for  multiple  sessions),  the  TacApps  design  team  was  able  to  gather  a  significant 
amount  of  user  insight.  Special  thanks  are  due  to  the  Soldiers  of  the  First  Infantry  Division  (1  ID)  and 
brigade  staff  for  taking  time  out  of  their  Command  Post  exercise  (CPX)  to  help  steer  future  U.S. 
Army  systems  development. 

Sessions 

Two  days  of  sessions  were  conducted  in  order  to  collect  feedback  from  both  division  and 
brigade  level  personnel. 

•  Day  1 :  session  with  1 1D  division  participants  held  at  the  division  headquarters  at 
Fort  Riley.  Sixteen  participants  left  their  CPX  event  to  join  in  2-hr  design 
sessions,  and  some  stayed  for  multiple  sessions. 

•  Day  2:  session  with  1 1D  brigade  participants  held  at  the  Mission  T raining  Center 
school  house.  Twenty-four  participants  left  their  continuing  CPX  event  to  join  in 
similar  2-hr  design  sessions,  and  one  stayed  for  multiple  sessions. 


Topics 


The  sessions  focused  on  the  user  interaction  topics  in  this  section. 

Roles  and  User  Identities 

Develop  user-facing  terminology  with  users  who  are  familiar  with  command  and 
Control  Systems  (C2)  but  are  unfamiliar  with  TacApps  and  the  common  operating  environment 
(COE). 


Language  of  Collaboration 

Develop  a  role-based  model  ensuring  the  user  interface  (Ul)  displays  data  in  ways 
that  are  intuitive  to  Soldiers  and  prevents  cognitive  overload  caused  by  screen  clutter. 

Product  and  Workspace  Management 

Elicit  user  input  on  management  of  live,  collaborative  products  versus  traditional 
models  and  impacts  on  language,  labeling,  training,  and  workflows. 
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Activities 

In  addition  to  an  introduction  to  the  TacApps  program,  each  design  session 
incorporated  a  variety  of  activities  that  include: 

•  Responding  to  short  video  clips  depicting  software  interactions. 

•  Completing  surveys  and  performing  card  sorting  exercises. 

•  Annotating  handouts  with  design  sketches. 

•  Providing  candid  feedback  in  small  group  sessions. 

Structure 

The  design  team  was  structured  into  facilitation  roles. 

•  Group  Guides  -  steered  activities  and  discussions. 

•  Scribes  -  recorded  the  Soldiers’  answers  and  comments. 

•  Leader  Facilitator  -  introduced  and  explained  each  new  concept. 

User  Feedback 

The  two  day  workshop  yielded  many  insights.  Several  major  themes  that  received  broad 
consensus  from  the  Soldiers  included: 

•  Validation  of  the  TacApps  collaboration  model  that  includes  a  shared  view  and  an 
individual  view  of  collaborative  data  and  visualizations. 

•  Similarly,  the  basic  TacApps  ownership  model  was  affirmed  by  participants, 
though  many  feature  requests  were  made  to  extend  the  Ul  visibility  of  ownership 
on  products  and  data. 

•  Many  users  believed  that  ownership  should  stop  at  the  board  level  and  not  extend 
to  visualizations  or  data. 

•  Voice  was  broadly  considered  to  be  an  integral  part  of  active  collaboration. 

•  Several  data-model  level  feature  requests  were  made,  including  a  running  history 
of  which  users  have  accessed  products  as  well  as  a  “who/when”  history  for 
changes. 

•  Participants  want  a  Ul  that  adheres  closely  to  Microsoft®  (and  other  common  web 
Ul)  conventions. 

•  TacApps  will  need  an  embedded  spellcheck  function  so  users  don’t  have  to  copy 
out  to  Microsoft®  Word. 

•  Users  rely  heavily  on  search  and  need  advanced  searching  tools  to  support  that 
primary  navigation. 
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•  Users  expect  TacApps  to  orient  them  to  their  unit’s  echelon  within  their  task 
organization  (org)  and  provide  easy  navigation  to  products  owned  by  units  above 
and  below  them  in  this  hierarchy. 

•  Mounted  users  need  the  full  suite  of  collaboration  tools  in  order  to  participate  in 
briefings  and  other  collaborative  activities  to  the  highest  degree  possible. 

•  Three-dimensional  (3D)  maps  are  not  used  collaboratively  and  are  seldom  used  at 
all  by  the  1 1D,  a  predominantly  ground  troop  force. 

•  System  interoperability  remains  a  primary  pain  point  for  users.  Moving  data 
between  [...],  AFATDS,  FBCB2,  and  DCGS  were  all  mentioned  many  times  as 
problems. 

•  Users  expect  a  family  of  social  media-style  interactions,  including  the  photograph 
identification  of  users,  online  presence,  and  tagging. 

•  The  participants  expressed  such  a  broad  range  of  ways  that  they  think  of 
collaboration  that  it  conclusively  justified  conducting  these  workshops  to  ensure 
that  TacApps  designs  a  model  that  combines  the  best  of  different  concepts  into 
one  that  will  best  meet  existing  user  expectations. 


Next  Steps 

The  TacApps  program  is  in  the  process  of  decomposing  the  feedback  from  this  workshop  into 
modifications  to  existing  design  documentation  where  appropriate,  and  into  information  that  will  be 
used  to  shape  required  design  concepts  that  are  currently  less  mature.  This  will  help  to  make  certain 
that  the  system  Uls  and  overall  user  experience  of  not  only  TacApps,  but  also  Mission  Command 
COE  v3  systems  in  general  will  be  built  to  include  design  patterns  and  human-computer  interaction 
expectations  revealed  by  the  interviewed  Soldiers  from  these  sessions.  The  program  also  plans  on 
conducting  subsequent  follow-up  workshops  and  TacApps  system  user  juries  that  place  real, 
working  software  in  front  of  the  same  1 1 D  participants  for  evaluation,  and  provide  tangible  evidence 
as  to  how  their  early  user  involvement  has  helped  to  noticeably  influence  the  design  of  the  system. 


DETAILED  REPORT 

Goals 

The  TacApps  design  workshop  was  designed  to  solicit  actionable  feedback  from  users.  The 
workshop’s  three  session  topics  were  chosen  to  prioritize  the  areas  in  which  user  input  would  most 
immediately  impact  design  direction.  The  three  topics,  and  their  contents,  were  selected  so  that: 

•  The  central  design  questions  in  the  topic  are  testable. 

•  Answers  to  those  central  design  questions  are: 

•  Crucial  to  the  overall  usability  of  TacApps. 

•  Timely  for  current  development  efforts. 
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Topics 


This  section  of  the  report  outlines  the  three  session  topics  used  in  the  workshop. 

Roles  and  User  Identities 

How  users  think  of  themselves  and  others  within  a  highly  collaborative  system  like 
TacApps  shapes  their  expectations  and  effective  use  of  the  system.  The  goal  of  this  topic  was  to 
understand  the  user’s  current  and  preferred  perceptions  of  themselves  within  a  system  context  as 
well  as  the  labels  they  expect  the  system  to  use  to  define  their  identity  and  the  identity  of  others 
across  different  contexts  (such  as  login,  finding  and  being  found,  and  forensic  investigation  of 
product  changes). 

Language  of  Collaboration 

How  users  understand  collaboration  and  its  many  possible  definitions  also  impacts 
their  use  and  interaction  with  the  system.  While  a  high-level  goal  of  this  topic  was  to  vet  the  current 
collaboration  model  design,  this  exercise  also  sought  to  reveal  participants’  workflows,  real-world 
collaboration  types,  and  approach  to  collaboration  in  general.  Activities  within  this  session  informed 
the  language  and  presentation  that  TacApps  will  use  in  collaborative  contexts. 

Product  and  Workspace  Management 

In  a  large-scale  collaborative  system,  potentially  thousands  of  users  will  be  creating 
and  using,  storing  and  labeling,  searching  for  and  finding  many  products.  The  goal  of  this  topic  was 
to  determine  preferred  or  potential  organizational  schemes  that  would  span  many  units  and  would 
satisfy  diverse  workflows  and  standard  operating  procedures  (SOPs).  Activities  were  designed  to 
reveal  patterns  in  how  the  users  organized  their  own  products  and  expected  to  find  other  users’ 
products.  The  session  also  uncovered  expectations  for  naming,  organization,  and  search 
capabilities  (fig.  1). 
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TacApps  design  workshop  team 


Hosts  and  facilitator 

Dexter  rewer 

Operations  SME 

Jim  Bell 

Operations  SME 

Janette  Knittel 

Designer 

Guides 

Dan  Dekowski 

Systems  Engineer 

Aaron  Lieb 

User  Experience  Lead 

Josh  Cauvel 

Technical  Lead 

Anthony  Kim 

Technical  SME 

Scribes 

Ross  Arnold 

Program  Technical  Lead 

Jason  Samuel 

Technical  Lead 

Chris  Doyle 

Systems  Engineer 

Pat  Forgione 

Systems  Engineer 

Chuck  Brown 

Designer 

Chick  Agnew 

Designer 

Lauren  Grim 

Designer 

Support 

Tim  Zirkel 

Army  Program  Office 

MAJ  Jacobs 

Army  Program  Office 

John  Raletz 

TRADOC 

LTC  Ayson 

KMO  1 1D 

LTC  Slagle 

1BDE  Dep  CO 

Note:  Subject  matter  expert  (SME),  U.S.  Army  Training  and 
Doctrine  Command  (TRADOC),  Lieutenant  Colonels  (LTC), 
Knowledge  Management  Officer  (KMO),  1st  Brigade  Deputy 
Commanding  Officer. 


Figure  1 

TacApps  design  workshop  team 


METHODOLOGY  OVERVIEW 

The  methodology  employed  in  the  design  workshop  was  developed  using  a  combination  of 
standard  user  experience  design  activities,  as  well  as  time-tested  experience,  with  the  unique 
challenges  presented  by  eliciting  user  feedback  in  a  military  environment.  For  example,  in  open 
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discussion  groups  and  user  juries,  one  or  two  highly  opinionated  or  particularly  vocal  subjects  can 
easily  dominate  the  direction  of  the  entire  room’s  conversations  and  feedback.  Other  variables  such 
as  rank,  seniority,  and  newness  to  the  organization  are  also  contributing  factors  that  can  result  in 
some  subset  of  the  group  receiving  an  unbalanced  amount  of  attention.  To  mitigate  this  dynamic,  the 
design  team  chose  the  paradigms  outlined  in  this  report. 

Focused  Small  Groups 

Each  session’s  participants  were  organized  into  small  groups  comprised  of  one  or  two  Soldiers,  a 
group  facilitator,  and  one  or  more  scribes.  Each  topic  was  preceded  by  an  explanation  of  the  key  TacApps 
concepts,  an  introduction  of  the  new  batch  of  Soldiers  to  the  workshop  team,  and,  finally,  a  brief  overview  of 
the  design  issue  types  being  focused  on  during  the  session.  Each  activity  was  then  conducted  at  the  individual 
table  groups,  such  that  four  different  tables  of  carefully-balanced  user  pairs  conducted  each  one 
simultaneously.  This  small  group  methodology  maximized  the  amount  of  relatively  unique  feedback  that  was 
gathered.  Similarly,  it  also  ensured  that  individual  participants’  rank,  conversation  patterns,  job  functions,  and 
the  table  facilitator’s  style  did  not  have  an  outsized  effect  on  the  workshop  results. 

Card  Sorting 

A  technique  called  “card  sorting”  was  used  that  allowed  the  design  team  to  very  rapidly 
present  participants  with  a  range  of  options  for  combining  and  prioritizing  the  cards.  The  design 
team  used  card  sorting  in  several  different  activities  during  the  workshop.  Card  sorting  addresses 
the  time  inefficiencies  of  asking  users  to  create  unique  answers,  and  narrows  the  range  of  feedback 
gathered,  for  insights  that  are  actionable  and  useful.  This  also  allows  the  scribes  to  note  not  only 
what  the  participant’s  final  “answer”  was  but  to  also  observe  their  thought  processes  and  behaviors 
as  they  arrived  at  those  answers. 

Limited  Exposure 

Finally,  the  design  team  used  a  technique  called  “limited  exposure.”  A  common  problem 
when  demonstrating  software  systems  to  users  is  the  breadth  of  understanding  that  they  must  gain 
in  order  to  provide  specific  feedback.  Time  is  wasted  and  findings  are  often  vague.  Using  quick 
videos  and  cards  instead  of  real  software  allowed  the  design  team  to  focus  on  a  small  subset  of  the 
system,  carefully  removing  any  details  that  weren’t  crucial  to  the  concepts  being  investigated  (fig.  2). 


Figure  2 

Participants  and  the  design  team  at  1 1D  headquarters 
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PARTICIPANT  DEMOGRAPHICS 

The  design  workshop  was  attended  by  a  diverse  mix  of  Soldiers  from  both  the  division  and 
brigade.  The  demographic  data  in  this  report  was  gleaned  from  surveys  completed  by  the 
participants.  This  information  provides  helpful  background  context  for  the  TacApps  design  team  to 
better  understand  the  responses  received  during  the  workshop.  Note  that  not  all  counts  total  the 
correct  number  of  participants,  as  some  Soldiers  didn’t  provide  answers  to  all  survey  questions. 

Participant  Ranks 

By  rank,  participants  ranged  from  Privates  (PVT)  to  LTC,  with  counts  indicated  in  figure  3. 
This  spread  represents  an  excellent  diversity  of  ranks  within  the  division  and  brigade  sessions, 
yielding  insights  from  many  different  perspectives  on  SOPs,  workflows,  and  collaboration. 


cm 


SSG  '  4 

5 


Note:  Major  (MAJ).  Chief  Warrant  Officer  (CW2),  Specialist  (SPC), 

Sergeant  (SGT),  Staff  Sergeant  (SSG),  Sergeant  First  Class  (SFC), 

Lieutenant  (LT),  Captain  (CPT) 

Figure  3 

Diversity  of  ranks  within  the  division  and  brigade  sessions 

Mobile  Operating  System  (OS)  Favored  by  Participants 

The  participants  reported  the  OS  used  on  their  personal  (non-Army)  mobile  devices.  The 
majority  of  participants  chose  iOS®  (Apple  Inc.’s  operating  system)  devices  when  given  an  option 
(fig.  4).  This  count,  while  not  a  strong  consensus,  reveals  a  preference  for  iOS®,  which  the  TacApps 
design  team  can  mine  for  reusable  design  patterns  (in  addition  to  Microsoft®). 
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Figure  4 

Mobile  OS  preferences 


Participant  Ages 

The  participants’  ages  ranged  from  19  to  43,  with  a  peak  in  the  25-and-under  category.  This 
is  a  decent  spread  of  ages,  though  there  is  an  obvious  peak  among  Millennial  (fig.  5). 


Figure  5 

Participant  age  range 

It  is  presumed  that  some  of  the  social-media-style  functionality  that  was  requested  comes 
from  within  the  youngest  age  bracket.  However,  those  requests  were  echoed  by  the  more 
experienced/higher  ranking  participants  as  well,  one  of  whom  remarked  “the  Army  is  a  young  place.” 
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Participant  Education  Levels 

The  education  data  collected  from  the  Soldiers  surveyed  indicates  that  the  majority  had  either 
already  earned  a  higher  education  degree  or  were  possibly  working  on  completing  one  (fig.  6). 


Figure  6 

Participant  education  levels 

Participant  Command  Post  of  the  Future  (CPOF)  Expertise 

The  participants  rated  their  CPOF  expertise.  There  is  a  large  number  at  the  lower  end  of  the 
expertise  curve.  This  chart  is  particularly  interesting  when  considered  in  conjunction  with  figure  7,  in 
which  the  participants  rated  themselves  as  generally  very  technically  advanced. 


Figure  7 

CPOF  range  of  expertise 
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Participant  Technical  Expertise 

Overall,  the  participants  ranked  themselves  as  being  technically  proficient.  The  confidence 
that  the  participants  indicated  in  their  general  technical  prowess  did  not  extend  to  CPOF  mastery,  as 
seen  in  figure  8. 


Figure  8 

Technical  level  of  expertise 


WORKSHOP  FINDINGS 
Session  1 :  Language  of  Collaboration 

The  TacApps  design  workshop  took  place  at  Fort  Riley  on  Tuesday  November  2,  2015,  from 
13:00  to  14:30.  This  session  covered  roles  and  user  identities. 

The  design  workshop  was  separated  into  three  sessions:  (1 )  language  of  collaboration,  (2) 
roles  [...],  and  (3)  product  management.  Each  of  the  three  sessions  was  run  each  day,  such  that 
there  was  a  total  of  two  language  of  collaboration  sessions  in  total,  etc. 

Each  session  began  with  room-wide  introductions  led  by  operations  SMEs,  followed  by  a 
brief  TacApps  system  overview,  presented  by  the  facilitator.  In  addition  to  session-by-session 
detailing  of  the  information  gathered,  this  section  includes  high-level  findings  (figs.  9  through  1 1 ).  All 
quotes  from  workshop  participants  have  been  anonymized  to  protect  the  Soldier’s  privacy. 
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•  Users  want  familiar  names,  not  made-up  terminology. 

•  “The  army’s  a  young  place,  so  if  we  can  use  a  ‘wall’  like  Facebook,  something  from 
outside  the  army,  that  would  be  good.” 

•  “Stick  to  language  you  can  find  in  social  media;  everyone  understands  that.” 

•  “(The  system)  has  to  be  user-friendly  on  all  levels;  keep  it  simple,  use  everyday  terms.” 

•  “Army  dudes  might  not  understand  words  like  ‘unique’.” 

•  TacApps  should  not  use  call  signs. 

•  “I’d  only  use  call  signs  on  the  radio.” 

•  “Call  signs  are  for  radios  only  and  when  you’re  deployed.” 

•  “Some  of  us  don’t  have  call  signs  and  some  may  share  call  signs.” 


Figure  9 

Findings,  terminology 


•  Users  want  to  review  a  history  of  who  has  recently  looked  at  products  they  own. 

•  “I  want  people  to  see  it  if  I  use  it,  and  retain  history.” 

•  Users  want  to  know  the  owner(s)  of  a  product. 

•  “(I’d)  like  to  see  a  list  of  most  recent  changes;  a  last  touched/modified  history...” 

•  “If  I’m  the  owner  of  the  product,  then  I  need  to  know  who  needs  it  so  I  can  tailor  it  for 
the  other  users.  If  someone  needs  my  product  specifically,  I’d  like  to  know  who’s 
looking  at  it.” 

•  Last  modified  who/when  value  on  products. 

•  (User)  says  a  last  modified  value  would  be  useful  to  help  tell  the  authoritative  data 
piece  from  a  duplicate.  (It  would  include)  the  time  it  was  changed  and  who  did  it. 
(User)  says  this  would  be  helpful  at  the  ‘visualization  and  effort’  level,  but  not  at  the 
data  level. 

•  “I  don’t  need  comments  on  why  they  changed  it  as  long  as  I  can  see 
who/when/contact  info  for  them.” 

•  “I’d  like  to  see  last  modified  by.  Attach  their  username  to  that  comment.” 

•  “When  he  changes  my  overlay,  the  system  should  track  who  made  the  change.” 

•  “That  way  you  wouldn’t  have  to  show  a  shift  change  brief.” 


Figure  10 

Findings,  data  presentation  needs 
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•  Integrated  voice  is  a  crucial  element  of  collaboration. 

•  “If  we’re  in  separate  locations,  we’ll  have  a  daily  scheduled  working  group  where  we 
use  common  map  and  voice  comms.  For  unscheduled  meetings,  we  generally  start  in 
chat  or  mere  chat,  highlight  something  on  the  map  and  keep  chatting.” 

•  “The  way  (I  designed  my  user  name  card  deck)  is  like  Ventrilo.  Ventrilo  lists  all  (users). 
You  can  look  and  find  and  talk.” 

•  “Even  a  hyperlink  to  contact  that  person  or  a  VOIP  number  would  be  good.” 

•  The  3D  maps  are  not  used  for  collaboration;  they  are  used  in  very  limited  circumstances  like  line- 
of-sight  analysis  and  planning. 

•  Users  like  the  idea  of  temporarily  granting  control  to  someone  else  during  a  real-time,  active 
collaboration  and  had  many  other  suggestions  on  how  permissions  could  be  made  easier  to  use. 

•  “Permissions  are  powerful  but  people  don’t  understand  them.” 

•  “You  don’t  want  issues  where  only  you  can  open  a  shared  document  and  it  stops 
someone  else  opening  it,  but  you  want  editing  permissions.” 

•  The  ability  to  quickly  and  easily  work  with  other  users  both  directly  and  indirectly  through  seeing 
their  work  is  desired. 

•  “(It  would)  be  cool  to  have  favorite  users.  Click  on  them  and  boom  -  see  their  stuff  or 
talk.” 

•  "Somehow  I  want  to  know  what  everyone  else  is  doing." 


Figure  1 1 

Findings,  use  of  collaboration 

After  introductions  and  the  TacApps  overview,  participants  began  the  language  session  with 
a  brief  video  on  the  TacApps  concept  of  data/work-product  ownership.  Participants  selected  as 
many  of  the  answers  (fig.  12)  that  they  felt  applied. 
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I  assume  that  these  rules  apply  to  a  map  that  I  find  in  a  folder: 

(Mark  as  many  as  you  want). 

□  "I  own  it" 

LJ  "I  don't  own  it" 

□  "I  can  open  it,  pan/zoom  it,  and  turn  layers  on  or  off' 

I  I  "|  can't  really  ijse  it  until  I  ask  the  ownerfor  permission" 

□  "If  I  change  the  location  on  the  map,  if  II  mess  someone  up” 

□  "Bob,  Joe,  and  MajorSmith  are  looking  at  a  different  view  of  the  same  map” 

□  "Nobody  knows  if  I  use  it  ornof' 

D  "Someone  else  might  see  what  I’m  doing,  but  they  won't  know  who  I  am" 

LJ  "The  map  will  say  who  it  belongsto" 

□  "I  can  add  data,  like  an  Event,  to  the  map"  1A 


Figure  12 
Map  checklist 

The  participants’  responses  are  detailed  in  figure  13.  Overall,  the  data  suggests  that 
participants’  expectations  generally  aligned  with  the  current  TacApps  ownership  model.  Only  one 
participant,  for  example,  believed  he  owned  a  product  simply  because  he  found  it  in  a  folder.  The 
majority  expected  they  did  not  necessarily  own  that  product  but  would  be  able  to  open  it,  navigate 
within  it,  and  use  layers.  These  expectations  are  in  line  with  the  current  design.  Since  only  one  user 
expected  that  navigating  the  map  location  would  “mess  someone  up,”  the  data  reveals  an  affirmation 
of  the  current  default  product  version,  which  is  to  include  live  data  capable  of  rich  collaboration  with  a 
severed  view  state  (in  the  style  of  the  CPOF  clone). 


These  rules  apply  to  a  map  I  find  in  a  folder 


I  own  it  I  don't  own  I  can  open  I  can't  really  If  I  change  Bob,  Joe,  Nobody  Someone  The  map  I  can  add 
it  it,  use  it  until  I  the  location  and  Maj  knows  if  I  else  might  will  say  whodata,  like  an 

pan/zoom  ask  the  on  the  map.  Smith  are  use  it  or  not  see  what  it  belongs  Event,  to 
it,  and  turn  ownerfor  it'll  mess  looking  at  a  I'm  doing,  to  the  map 

layers  on  or  permission  someone  different  but  they 

off  up  view  of  the  won't  know 

same  map  who  I  am 


Figure  13 

Compilation  of  map  usage  data 
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Users  showed  more  uncertainty  about  whether  non-owners  can  add  data  to  the  map; 
however,  that  indicates  a  need  for  clarifying  this  aspect  of  the  TacApps  ownership  model. 

Next,  participants  watched  a  brief  video  on  the  T acApps  concept  of  a 
collaboration/composition  board.  Participants  circled  answers  (fig.  14)  that  matched  their 
understanding  of  how  boards  should  work.  This  card  revealed  the  Soldiers’  ideas  of  the  basic 
relationships  of  collaboration. 


Circle  the  best  answer  to  complete  each  statement: 

1.  Joe  and  Bob  ore  /  are  not  collaborating. 

2.  Bob  and  Major  Smith  are  /  are  not  collaborating. 

3.  The  three  of  them  are  /  are  not  collaborating. 

4.  Major  Smith  needs  to  know  /  doesn't  care  that  someone  is 
looking  at  /  using  a  board  he  owns. 

IB 


Figure  14 

Basic  understanding  of  the  usage  of  the  term  “collaboration”  within  CPOF 

Analysis  of  these  responses  indicates  that  participants  were  generally  more  likely  to  answer 
affirmatively  (basically  meaning  that  some  groups  of  the  video’s  users  were  collaborating),  rather 
than  negatively  (fig.  15).  However,  there  was  not  a  strong  consensus  for  most  questions,  which 
illustrates  the  fundamental  differences  in  how  individuals  view  collaboration.  For  some  participants, 
the  events  in  the  video  met  their  definition  of  collaboration,  while  others  saw  them  differently.  A 
perennial  challenge  of  meeting  the  expectations  of  users  of  a  collaborative  system  is  addressing  the 
myriad  ways  that  collaboration  can  be  understood. 
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Joe  and  Bob  Joe  and  Bob  Bob  and  Maj  Bob  and  Maj  The  three  of  The  three  of  Maj  Smith 
are  are  not  Smith  are  Smith  are  not  them  are  them  are  not  needs  to 

collaborating,  collaborating,  collaborating,  collaborating,  collaborating,  collaborating,  know  that 

someone  is 
looking 
at/using  a 
board  he 
owns. 


Maj  Smith 
doesn't  care 
that 

someone  is 
looking 
at/using  a 
board  he 
owns. 


Figure  15 

Statements  that  describe  map  collaboration  video 

Next,  they  filled  out  a  questionnaire  (fig.  16),  selecting  as  many  answers  as  they  thought 
applied.  This  card  revealed  more  specifics  about  how  ownership  concepts  would  apply  in  a 
collaborative  workflow.  The  results  are  shown  in  figure  17. 


I  assume  that  these  rules  apply  to  a  board  that  I  find  in  a  folder: 

(Mark  as  many  as  you  want). 


□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 

□ 


"I  own  it" 

"I  don't  own  it” 

"I  can't  really  use  it  until  I  ask  the  owner  for  permission" 

"If  I  zoom  or  pan  on  the  map  in  this  board,  it'll  mess  someone  up" 
"Everyone  else  locking  at  this  board  has  a  different  view  of  the  same  map" 
"Nobody  knows  if  I  use  it  or  not" 

"Someone  else  might  see  what  I'm  doing,  but  they  won't  know  who  I  am" 
'The  board  will  say  who  it  belongs  to" 

“I  can  add  data,  like  another  map,  to  the  board" 


1C 


Figure  16 

Questions  indicating  how  ownership  concepts  would  apply  in  a  collaborative  workflow 
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I  own  it  I  don't  own  I  can't  really  If  I  pan  or  Everyone 
it  use  it  until  I  zoom  the  else  looking 
ask  the  map  in  this  at  this 
owner  for  board,  it'll  board  has  a 
permission  mess  different 
someone  view  of  the 
up  same  map. 


Nobody  Someone  The  board  I  can  add 
knows  if  I  else  might  will  say  who  data,  like 
use  it  or  not  see  what  it  belongs  another 
I'm  doing,  to  map,  to  the 
but  they  board 

won't  know 
who  I  am 


Figure  17 

Results  of  how  ownership  rules  apply  to  a  board  found  in  a  folder 

Next,  participants  watched  another  brief  video  on  the  TacApps  concept  of  visualizations  with 
two  view  states:  a  shared  view  and  an  individual  view.  Then,  they  circled  as  many  terms  for  the  two 
view  states  as  they  liked  on  the  questionnaire  (fig.  18),  which  revealed  the  participants’  overall 
terminology  preferences.  The  results  are  shown  in  figure  19. 


Which  terms  make  the  most  sense  to  describe  the  views  shown  in  the  video, 
where  Joe  and  Bob  could  choose  to  share  a  view  or  not?  Circle  as  many  as 
you  like. 


"Explore" 

"In  Sync" 

"Unique" 

"Matched" 

"Navigate" 

"Co-Navigate" 

"Go  alone" 

"Keeptime  with" 

"Solo" 

"Lock-Step" 

"Separate" 

"Reconnect" 

"Drive  Solo” 

"Drive  Together" 

"Solo  View" 

"Unked" 

“Shared  View" 

"Unison" 

Figure  18 

Terminology  preferences 
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Figure  19 

Naming  of  individual  versus  shared  views 

Next,  the  TacApps  design  team  introduced  the  concept  of  onboarding,  a  software  design 
concept  that  orients  new  users  to  a  system’s  tools  and  interactions.  Participants  viewed  examples  of 
various  onboarding  methods  used  in  other  contexts  (websites,  development  tools,  and  social  media) 
and  then  answered  which  revealed  the  most  important,  initial  tools  that  TacApps  must  deliver  to  new 
users  (fig.  20).  The  results  were  weighted  based  on  the  order  in  which  the  items  were  listed  by  each 
user. 
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Figure  20 

Most  important  initial  tools 
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Participants  also  filled  out  card  1 F,  which  gathered  information  about  which  elements  of  the 
onboarding  examples  seemed  helpful  or  not  helpful  to  the  Soldiers  in  a  TacApps  context  (fig.  21). 
Classroom-wide  discussion  followed,  and  participants  were  encouraged  to  explain  the  “why”  behind 
their  answers. 
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4 

3 

2 

1 
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Figure  21 

Onboarding  example  likes  and  dislikes 

Session  2:  Roles  and  User  Identities 

The  TacApps  design  workshop  was  held  at  Fort  Riley  on  Tuesday  November  2,  2015,  from 
13:00  to  14:30.  This  session  covered  roles  and  user  identities. 

The  goal  of  this  session  was  to  clarify  the  best  method  of  identifying  users  in  the  TacApps 
system.  The  sessions  led  the  users  through  a  variety  of  activities  aimed  at  presenting  use  cases  in 
which  the  user  name  is  important  and  asked  the  users  to  identify  traits  and  characteristics  required  of 
the  user  name.  The  users  were  presented  with  10  traits  that  included:  (1)  rank,  (2)  first  name,  (3)  last 
name,  (4)  middle  name,  (5)  role,  (6)  staff  function,  (7)  warfighter  function,  (8)  call  sign,  (9)  special 
skills,  and  (10)  asked  them  to  create  a  user  name.  After  presenting  the  use  cases,  the  users 
repeated  the  exercise  to  check  if  there  were  any  differences  in  their  opinions  (figs.  22  and  23). 
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Online  User  Presence 

•  Knowing  whose  eyes  are  on  a  product  in  real  time  is  necessary  during  briefings,  and 
helpful  but  not  necessary  in  other  contexts. 

•  “If  I’m  the  owner  of  the  product  then  I  need  to  know  who  needs  it  so  I  can  tailor 
it  for  the  other  users.  If  someone  needs  my  product  specifically,  I’d  like  to 
know  who’s  looking  at  it.” 

•  “Knowing  who’s  actively  looking  at  your  work-product  like  in  Adobe  Connect  is 
not  totally  needed.  In  briefing  yes,  but  not  otherwise.” 

•  “Presence  would  be  great  here  -  if  they’re  at  the  desk  or  not.  Like  on  Outlook.” 

•  (User)  expects  to  see  a  user’s  presence  if  they’re  online  -  grayed  out  for 
offline  and  green  for  online. 


Identification  and  Tagging 

•  A  number  of  users  favored  the  use  of  photograph  identification  for  each  user,  others 
thought  of  this  as  optional  and  not  sufficient  to  fully  identify  others. 

•  “Picture  with  a  name  would  be  huge.” 

•  “A  picture  would  be  helpful.  Rank,  last  name,  role  under  the  pic.  We  do  a  lot 
of  seating  charts  and  could  narrow  down  by  role.” 

•  “It  would  be  helpful  to  put  a  face  to  the  name  plus  a  role.” 

•  “It  could  come  from  the  CAC  card  pic.” 

•  “If  I  saw  actual  pictures,  it  wouldn’t  be  enough  to  identify  a  person.” 

•  Rank  and  name,  and  the  picture  (not  required)  could  also  use  a  rank  symbol 
instead  of  face. 

•  Users  rely  heavily  and  primarily  on  search  over  other  navigational  techniques  when 
searching  for  unfamiliar  and  infrequently  used  products.  Also  they  like  tagging  because 
it  enriches  search. 

•  “I  don’t  need  to  see  all  users  because  I  have  search  functions.” 

•  “(User)  would  be  interested  in  ‘I’m  an  expert  in  (building  out  the  COPs)”  type 
expert  tagging.” 

•  “There’s  value  in  taxonomy  -  in  tagging  things  CUOP,  FUOP,  because  search 
is  incredibly  difficult.  Searching,  with  filters,  across  the  entire  system  of  apps 
would  be  incredibly  valuable.  I  would  say  that  should  be  a  requirement, 
saying  that  as  a  knowledge  manager.” 

•  "Outlook  is  searchable  by  name,  I  want  to  be  able  to  search  by  various 
functions:  'all  people  in  BRAVO  Company'  (dropdown).” 


Figure  22 

Findings,  social  media  expectations 
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•  Users  want  a  workflow  that  combines  fast  collaboration  plus  eventual  ‘official’  approval  to 
ensure  trustworthiness  of  information. 

•  “Initially,  unit  SOP  (should  dictate  organizational  workflows)  to  get  grassroots 
appropriateness.  Then,  allow  a  system  to  push  that  (organizational  system)  out  as  a 
pattern  that  other  units  can  adopt.” 

•  “At  some  point  if  you  were  able  to  take  the  draft  and  the  certified  person  could 
approve  it...  it  could  be  a  workflow.” 

•  “I  would  like  to  see  a  feature  where  I  can  approve  or  deny  additions/edits.  That’s  how 
it  is  in  AFATDS,  through  recommendations.  For  us,  wrong  information  causes  a  lot  of 
problems.” 


•  Users  want  to  take  someone  else’s  product  to  use  as  a  head-start  template,  and  then  modify  as 
needed. 

•  In  briefing,  a  common  template  should  be  shared  with  the  subordinates. 

•  Warfighter  functions  would  create  templates  for  their  specific  functions. 


•  Users  want  to  label  work  products  by  status,  such  as  ‘draft.’ 

•  “While  a  product  is  still  being  built,  no  one  should  be  able  to  look  at  it  because  they’ll 
get  false  information.” 

•  “We  need  a  way  to  clearly  label  a  product  as  ‘draft.’  Near  the  classification  border 
maybe.” 


•  Users  want  a  way  to  add  things  to  products  they  don’t  own. 

•  “If  an  owner  doesn’t  put  a  layer  on  that,  then  I’d  have  to  email  that  request.  It  could  be 
up  to  12  hours  before  I  read  an  email,  so  requesting  a  layer  or  permission  can  take 
very  long.” 

•  “He  sends  the  changes  to  me,  unless  he  gives  me  permission  or  lets  me  contribute.” 


•  Users  want  the  ability  to  easily  save  TacApps  boards  as  PowerPoint  presentations. 

•  On  copying  and  pasting  with  PowerPoint  -  "screen  grabs  on  a  paste  board  look  like  a  2 
year-old  did  it." 

•  “(I’m)  doing  SigAct  briefs  in  PowerPoint,  requested  by  my  Commander.” 

•  (User’s)  unit  has  had  to  recreate  entire  CPOF  products  in  PowerPoint. 

•  (User)  suggests  we  allow  a  user  to  type  their  role  in  immediately  but  then  let  the  next 
level  user  (supervisor)  verify/screen  it. 


Figure  23 

Findings,  workflow  needs 
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There  were  eight  Soldiers  from  the  division  who  participated  in  this  session.  Another  eight 
Soldiers  from  the  brigade  participated  in  an  identical  session  on  the  next  day. 

Participants  were  given  card  2A,  which  asked  them  to  write  down  the  user  name  they’d  want 
to  have  in  the  TacApps  system.  This  was  followed  by  a  room-wide  discussion  in  which  the  Soldiers 
explained  their  reasoning  behind  their  user  name  selection. 

Next,  the  participants  were  given  a  series  of  10  cards,  each  with  a  possible  component  of  a 
TacApps  user  identity  (such  as  role,  rank,  last  name,  etc.)  and  were  asked  to  sort  the  cards:  most 
crucial  identity  elements  first,  followed  by  less  crucial  elements,  and  unimportant  elements  were 
discarded.  This  activity  encouraged  participants  to  think  of  how  other  system  users  would  best  be 
able  to  find  them. 

For  the  next  exercise,  participants  labeled  and  sketched  on  a  printed  handout  (handout  2-1). 
This  activity  revealed  the  way  participants  expect  the  TacApps  system  to  identify  a  whole  range  of 
U.S.  Army  users. 

Then,  participants  were  told  that  the  TacApps  system  would  have  a  list  of  each  user’s  “most 
useful  contacts.”  Participants  were  asked  to  design  that  list  on  handout  2-2:  who  would  be  on  it,  how 
would  they  be  identified,  and  in  what  scenarios  would  participants  find  and  use  this  contact 
information  (fig.  24). 


—  TACApps 

n  Product  Drive 
@  Us*re 
#  APP5 

®  Settings 
Logout 


All  Users: 


Figure  24 

Most  useful  contacts  list 

Then,  participants  were  given  a  blank  card  (card  2L).  They  were  told  that  a  controversial 
comment  had  been  made  on  a  piece  of  data  in  the  TacApps  system,  and  they  were  tasked  with 
finding  out  who  made  the  comment,  how  they  would  expect  to  locate  the  commenter,  and  what  the 
system  should  provide  in  the  way  of  identifying  users  in  this  context.  This  exercise  revealed  the 
participants’  ideals  for  forensic  discovery  in  TacApps. 
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Next,  participants  were  given  a  series  of  four  identities,  each  formatted  differently  (fig.  25). 


John  M.  Smith 


Battle  Captain  from  2BDE 


CG  from  Division 


Note:  Commanding  General  (CG) 

Figure  25 

Example  user  identities 

The  goal  was  to  see  how  the  participants  would  go  about  locating  these  people  in  TacApps. 
The  TacApps  design  team  facilitated  this  paper  prototype  navigation  in  real  time  with  the  participants 
-  providing  printed  handouts  of  system  menus  and  locations  as  needed  and  recording  the 
participants’  expected  search  paths. 

For  the  final  exercise  in  this  session,  participants  were  asked  to  return  to  the  card  sorting 
activity  they  completed  previously:  10  cards,  each  with  a  possible  component  of  a  TacApps  user 
identity  (such  as  role,  rank,  last  name,  etc.).  Once  again,  participants  sorted  the  cards:  most  crucial 
identity  elements  first,  followed  by  less  crucial  elements,  and  unimportant  elements  were  discarded. 
By  returning  to  this  activity,  the  TacApps  design  team  was  able  to  compare  each  participants  “gut 
reaction”  to  their  more  nuanced  answer  after  completing  several  exercises  about  the  different  ways 
that  user  identities  will  appear  in  TacApps.  Generally  speaking,  though  individual  users  frequently 
changed  the  order  of  their  answers,  there  was  very  little  difference  between  the  first  and  second  card 
sorts  when  reviewed  in  aggregate  (fig.  26).  The  card  sort  was  followed  by  a  room-wide  discussion  in 
which  participants  shared  their  insight  about  the  design  of  TacApps  user  roles  (fig.  26). 


Figure  26 

Role  attributes,  relative  popularity  -  exercise  results 
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Examples: 

•  1AD  G-2  ACofS  MAJ  Parker  (Division  Intelligence  Assistant  Chief  of  Staff,  Major 
Parker). 

•  1/1AD  BTL  CDR  COL  Wayne  (Brigade  Commanding  Officer,  Colonel  Wayne). 

•  1/1-1AD  S-9  AN  SPC  Banner  (Battalion  Civil  Affairs  Analyst,  Specialist  Banner). 
Session  3:  Product  and  Workspace  Management 

The  participants  were  given  a  brief  overview  on  product  management  in  TacApps,  specifically 
the  concept  and  implementation  of  the  product  drive  (where  their  work-products  would  be  stored, 
organized,  found,  and  created).  They  were  shown  a  basic  wireframe  image  of  what  the  product  drive 
might  look  like  (figs.  27  and  28). 


Users  want  their  location  in  the  task  org  to  be  the  starting  point  for  folder/product/person  organization. 
Also,  they  want  to  easily  be  able  to  navigate  one  echelon  up  and  one  echelon  down  from  their  own. 

•  “I  would  like  to  see  all  the  people  doing  logistics.  Not  just  mine,  but  my  division  first,  then  like 
Outlook  you  can  go  to  Global.  But  at  first,  the  ones  I  contact  the  most.  Why  would  I  need  to 
see  a  Fires  guy?” 

•  On  hierarchy  organization  -  “One  up,  one  down!” 


Users  want  to  name  the  folder  of  their  other  user  contacts  ‘favorites;’  it  should  be  a  combination  of 
automatically  added  names  and  the  ability  to  manually  add/remove  names. 

•  “Battalion  people  separate  from  my  units  like  SI ...  but  when  I  click  SI  it  bleeds  out  the  people 
in  there.” 

•  "It  would  be  nice  if  the  system  determined  who  I  contact  most  (top  5,  in  the  past  6  months, 
based  on  frequency),  and  add  to  my  contact  list." 

•  “Have  people  in  your  unit  separated  from  higher  ups  because  I’d  get  commands  from  them... 
like  notifications.  Then  have  other  shops.” 


Users  provided  no  strong  consensus  on  how  TacApps  should  identify  them. 

•  “First  and  last.  Easy  to  remember.  My  role  is  not  as  important.” 

•  “Role  is  important  whereas  identity  is  not  important.” 

•  “Enlisted  don’t  use  first  names.  That’s  for  officers.  It  wouldn’t  help  me  to  see  a  first  name  in 
a  computer.” 


Figure  27 

Findings,  information  organization 
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Mounted/dismounted  users  require  the  full  suite  of  briefing  and  collaboration  tools. 

•  “For  the  mounted  environment,  it’s  incredibly  important  to  have  collaboration.” 

•  “The  biggest  thing  now  is  getting  the  soldier  level,  like  at  platoon,  to  put  info  right  into  the 
system  directly,  not  swivel  chair  FBCB2  info  and  then  enter  it  into  CPOF.” 

•  “We  need  to  dialog  and  command  from  a  mounted  platform  too,  to  give  a  briefing.” 

•  “In  a  Bradley,  I  would  be  both  sharing  and  the  one  briefing  for  things  like  route  clearance.” 

•  On  briefing  in  a  mounted  environment  -  (mobile)  commanders,  G3s,  S3s-it  would  be  a  great 
benefit  to  them.  If  the  company  was  doing  some  sort  of  a  sync,  and  had  their  platoons  out, 
this  would  be  a  good  way  to  communicate.  Being  able  to  notify  the  Battle  Captain  when 
(there  is)  an  update  is  important.” 

•  “The  battlefield  changes  every  minute.  There  is  no  front  line.  The  lowest  soldier  is  (only) 
equipped  with  cell  quality  radio.  He  can  add  an  IED  on  site  on  his  phone,  and  feed  Brigade, 
Battalion,  Division.” 


Figure  28 

Findings,  mounted/dismounted  needs 

The  first  activity  asked  the  participants  to  label  a  TacApps  folder  that  would  be  accessible 
only  by  them,  not  others.  They  were  given  a  series  of  10  cards,  each  of  which  contains  a  term  (such 
as  “my  folder”)  or  an  element  of  a  user  identity  (such  as  “last  name”)  as  well  as  blank  cards  for  the 
participants  to  provide  creative  suggestions.  They  ordered  the  cards,  as  many  as  needed,  as  they 
would  label  their  TacApps  folder. 

The  next  exercise  (handout  3-2)  asked  the  participants  to  sketch  what  they  would  expect  to 
see  under  the  TacApps  “groups”  menu  item,  how  the  folders  would  be  organized,  which  method  of 
separating  the  U.S.  Army’s  many  users  into  groups  they  would  want  to  see,  and  what  would  be  most 
helpful  (fig.  29). 
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Figure  29 

TacApps  “groups”  menu  item  example 

Then,  participants  were  asked  about  their  current  use  of  CPOF  in  order  to  provide 
background  context  to  their  answers  throughout  the  session  and  to  provide  the  TacApps  design 
team  with  do’s  and  don’ts  moving  forward.  This  conversation-based  exercise  yielded  much 
information  about  the  Soldiers’  real-world  workflows  and  system  use. 

In  the  next  activity,  the  TacApps  design  team  drilled  down  into  the  participants’  CPOF  usage, 
more  specifically,  information  and  product  sharing. 

Then,  the  next  conversation  drilled  into  the  participants’  expectations  and  understanding  of 
how  TacApps  should  handle  authoritative  data,  such  as  a  common  operating  picture  (COP)  map,  or 
boundary  graphics.  This  session  yielded  more  real-world  insight  into  the  Soldiers’  needs. 


DESIGN  AFFIRMATIONS 


Individual  View,  Shared/View 

The  most  affirmed  design  model  was  the  individual  view  versus  a  shared  view  collaboration 
pattern.  Even  before  the  design  was  introduced  to  participants,  many  of  them  described  their  ideal 
functionality  in  a  way  that  foreshadowed  the  design  they  would  later  be  shown: 

•  "I  wouldn't  mind  a  preview  when  I  click  on  a  work  product." 
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•  "I  want  the  ability  to  look  at  a  different  view.  Default  starts  at  the  same  reference 
for  everyone.  When  I  go  in,  I  want  to  see  the  same  view  as  Major  Smith,  but  I 
want  the  ability  to  dork  with  it." 

•  “(It’s)  really  nice  to  be  able  to  move  the  map  around  without  affecting  the  brief  or 
who  is  working  with  the  map.” 

•  “In  CPOF,  we’ve  had  same  logins  battling  workspace.  In  this  third  video,  I  like 
making  changes  that  all  see  without  impacting  anyone.” 

•  “I  don’t  want  them  to  be  able  to  move  my  map  whenever  they  want,  but  it  would  be 
useful  for  someone  to  be  able  to  highlight  or  pan/zoom  when  they  need  to  point 
out  something.” 

Folder  Design 

The  folder  structure  was  broadly  considered  intuitive,  even  though  there  was  a  lot  of  disparity 
on  which  levels  of  U.S.  Army  hierarchy  should  be  placed  and  where.  The  similarity  to  Microsoft®  was 
considered  a  plus  by  the  majority  of  participants: 

•  "I  wouldn't  mind  a  preview  when  I  click  on  a  work  product." 


Mounted 

The  folder  structure  was  broadly  considered  intuitive,  even  though  there  was  much  disparity 
on  which  levels  of  U.S.  Army  hierarchy  should  be  placed  and  where.  The  similarity  to  Microsoft®  was 
considered  a  plus  by  the  majority  of  participants. 

Rather  than  being  a  secondary  concern,  participants  expressed  the  desire  to  have  a  full  suite 
of  collaborative  tools  for  mounted  environments  to  attend  and  provide  briefings: 

•  “As  soon  as  you  get  on  CPOF  you’re  not  mobile.  You  need  to  be  mobile.” 

•  “For  the  mounted  environment,  it’s  incredibly  important  to  have  collaboration.” 

•  “In  a  mounted  configuration,  both  passive  and  active  collaboration  are  needed.” 

•  “We  need  to  dialog  and  command  from  a  mounted  platform  too,  to  give  a  briefing.” 

•  “For  example,  in  a  Bradley,  I  would  be  sharing  the  one  briefing  for  things  like  route 
clearance.” 

•  “On  a  computer,  (this  is)  fine.  Out  in  the  desert,  less  is  fine.” 

Briefing  Concept 

Early  concepts  for  a  briefing  application  have  included  functionality,  such  that  a  user  who  had 
placed  his  board  in  the  shared  briefing  could  make  data  changes  to  his  local  version  of  the  board, 
confident  that  those  changes  (to  live  data)  would  be  near-instantaneously  included  on  the  brief: 

•  “If  you  hear  a  CG  give  feedback  during  a  brief,  you  want  to  be  able  to  fix  your 
products  right  away  before  they  get  to  yours.” 
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•  “(It’s  important  during  a  brief)  to  know  whose  eyes  are  on  the  brief.” 

•  “Briefing  requires  a  dialogue.” 

•  “Briefing  is  really  the  only  context  you  want  to  know  if  you’re  collaborating  with 
somebody  right  now.” 

•  “Active  is  briefing,  it’s  dialog.  Passive  is  sharing.” 

•  “Some  places,  before  the  briefing  starts,  the  group  will  replicate  the  brief  to  protect 
themselves  from  themselves.  A  frozen  product  before  briefing  can  be  important.” 

•  A  “who  I’m  sharing  with”  (banner  of  some  type)  would  help  to  reinforce  privacy 
when  it  says  “nobody.” 

Storing  Data  in  “My  Folder" 

•  “(User)  stores  his  products  in  his  ‘CPOF  user,’  not  in  shared  products.” 

Folder/Groups  Design 

•  “Go  to  groups  and  find  HQ.  Would  then  need  standard  role  names,  which  would  update  if 
changed.” 

Insisting  on  Simplicity 

•  “Limitations  are  important.  Keeping  customizability  to  a  minimum  is  key.  For  example, 
being  able  to  turn  friendly  units  red  (just)  because  your  division  likes  red  (is  a  problem).” 

•  “But  the  systems  don’t  do  detailed  staff  planning  because  they  can’t  replicate  the 
precision  of  a  mouse  (for  mounted).” 

Tooltips  for  Instruction 

•  Tooltips  would  help  the  user.  They  prefer  that  type  of  interaction  teaching  within  the 
application. 

Attaching  Chat/Email/other  Messages  to  Products 

•  “Being  able  to  share  with  another  person  to  ask  for  feedback  (is  important).” 

•  (The  user)  describes:  “A  link  to  data  would  make  it  a  lot  easier  if  you  could  just  send  data 
to  a  distribution  list  then  click  on  the  shared  locations  and  have  it  automatically  open  up.” 

Right-click  Menus 

•  “Have  a  simple  right-click  with  options  like  ‘share  product’.” 
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Limited  Need  for  Three-dimensional  Map  Collaboration 

•  “(The  user)  has  never  heard  of  any  need  to  do  live  3D  collaboration  from  a  mounted 
environment.  He  has  seen  tilting  the  2D  map  to  see  terrain  but  has  never  used  live  3D 
map  collaboration  at  all.” 

•  “(User)  is  not  interested  in  3D  map  collaboration  capability.” 

•  “(User)  agrees  that  3D  is  not  necessary  for  collaboration.” 

•  “3D  (maps)  don’t  get  much  use  except  for  line  of  sight  analysis  requirements,  and  to 
appreciate  3D  terrain  during  planning.” 

•  “In  a  briefing  you  can  get  away  without  (3D  functionality).” 

•  “I  don’t  touch  the  3D  map.  I  don’t  see  any  benefits  to  it  -  the  same  with  animation  and 
time  stuff;  it  has  no  added  value.” 

Fixed  Board  Layouts 

•  “But  I’ll  also  like  the  new  rigid  layout  of  the  board  and  how  it  doesn’t  let  you  lose  things 
under  things  and  get  cluttered.” 

Ownership  Basics 

•  “The  people  that  we  allow  to  alter  (our  products)  is  a  very,  very  small  list.” 


DESIGN  CHALLENGES 


Ownership  Details 

Many  to  most  participants  believed  ownership  stops  at  the  board  level  (i.e.,  if  one  owns  a 
board,  they  own  everything  within  that  board)  and  failed  to  understand  the  composability  requirement 
that  visualizations  and  data  contained  within  a  board  frequently  will  have  owners  who  are  not  the 
board  owner.  There  was  also  a  lot  of  feedback  that  permissions  are  difficult  to  discover/understand, 
and  it  is  likely  that  a  familiar  and  thorny  design  challenge  needs  to  be  overcome. 

•  “With  a  board  (there  are)  only  two  answers:  either  you  own  it  or  you  don’t.  If  you 
own  it,  you  can  change  it.” 

•  “(I’d)  like  to  see  permissions  stop  at  the  pasteboard  level  -  not  go  to  the 
visualization/frame  (map)  level.” 

Personal  Folder 

While  some  participants  appreciated  the  idea  of  a  personal  folder,  some  challenged  its 
necessity. 


•  “I  don’t  think  anybody  needs  100%  privacy.” 
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•  “(I)  see  no  reason  why  invisible,  ‘nobody  can  see  it’  privs  would  ever  be  needed. 

(I’m)  okay  with  everybody  being  able  to  view  everything.” 


Task  Organization 

Information  and  access  organized  around  the  individual’s  place  in  the  task  org  was  a  near- 
universal  request,  but  it  remains  unclear  at  the  program  level  if  there  are  resources  to  ensure  that 
task  orgs  will  be  available  to  most  TacApps  users. 

Need  for  Authoritative  Products 

•  “Everybody  puts  stuff  in  the  tree  viewer,  under  shared  products.  It’s  starting  to  pile  up. 
Why  are  there  four  COPs?  There  should  be  one!  People  are  making  templates...” 

•  “Every  time  you  change  a  role,  the  system  should  red-border-big-asterisk  REQUIRE  that 
you  confirm  or  update  your  phone  number.” 

•  “(A)  last  modified  value  would  be  useful  to  help  tell  the  authoritative  data  piece  from  a 
duplicate.  (It  would  include)  the  time  it  was  changed  and  who  did  it.” 

Collaboration  Ideas 

General  definitions  of  “what  is  collaboration”  varied  wildly,  making  it  difficult  to  deliver  a 
collaboration  model  that  meets  a  majority  of  users’  expectations. 

•  “I  don’t  think  they  are  talking/collaborating  because  they’re  just  observing...  It 
doesn’t  look  like  they’re  working  together...  (They  are  working  together)  if  they  are 
talking.” 

•  “What  does  collaboration  mean?” 

•  “Users  are  not  collaborating  because  only  one  person  is  editing;  the  other  users 
are  only  watching.” 

•  “I  think  of  collaboration  as  (a  number  of)  people  equally  building  something 
together.” 

•  “Collaboration  means  you  have  the  same  rights.” 

System  Interoperability 

System  interoperability  remains  a  primary  pain  point  for  many  to  most  users.  Moving  data 
between  [...],  Advanced  Field  Artillery  Tactical  Data  System  (AFATDS),  Force  XXI  Battle  Command 
Brigade  and  Below  (FBCB2),  and  Distributed  Common  Ground  System  (DCGS)  were  all  mentioned 
many  times  as  problems. 

•  “The  biggest  thing  now  is  getting  the  Soldier  level,  like  at  platoon,  to  put  info  right 
into  the  system  directly,  not  swivel  chair  FBCB2  info  and  then  enter  it  into  CPOF.” 

•  "Are  you  guys  trying  to  make  (TacApps)  talk  to  JCR  or  BFT?" 
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•  “Whatever  else,  if  what  you  develop  can’t  talk/share  to  the  other  AMDS  programs, 
it’s  useless.  Also,  individual  programs  (like)  DSGS,  AFATDS,  JTR,  etc.,  which 
operate  in  isolation  defeat  the  concept  of  common  operating  picture.” 

User  Identity 

There  was  no  clear  consensus  on  whether  TacApps  users  should  login  by  name  or  role. 

Users  need  to  find/collaborate  with  both  individuals  as  well  as  people  currently  in  roles: 

•  “First  and  last.  Easy  to  remember.  My  role  is  not  as  important.” 

•  “Role  is  important  whereas  identity  is  not  important.” 

•  “Enlisted  don’t  use  first  names.  That’s  for  officers.  It  wouldn’t  help  me  to  see  a 
first  name  in  a  computer.” 

•  “Some  of  us  don’t  have  call  signs  and  some  may  share  call  signs.” 

•  “Unit  is  critical;  rank  is  nice  to  have.” 

Fundamental  System  Types 

Participants  weren’t  clear  on  the  delineation  between  visualizations  and  data  (for  example, 
they  didn’t  know  what  “efforts/folders”  were),  which  makes  it  likely  that  they  could  remain  confused 
by  any  system-level  behavioral/interaction  differences  between  the  two. 

•  (User)  doesn’t  think  of  efforts  as  data. 

Product-centric  Organization 

Most  users  have  an  extremely  product-centric  view  of  the  system  (for  products  like  the  COP, 
etc.)  rather  than  a  people/group,  date/time,  or  other  organizational  method.  This  method  may  not 
necessarily  support  a  product-centric  worldview: 

•  “You  haven’t  mentioned  staff  roles  once.  You’ve  mentioned  high-level  products 
like  the  COP  (design  team  member  noting  the  participant’s  focus).” 

Adding  to/Changing  Unowned  Products 

Users  want  to  be  able  to  “push”  products;  to  put  a  product  in  a  place  that  they  don’t  have 
ownership  of,  and  then  allow  the  owner  to  accept  or  reject  those  changes: 

•  "I  don't  want  to  have  to  ask  permission." 

•  (The  user)  wouldn’t  mind  others  looking  at  a  product  he’s  working  on  updating  so 
that  they  can  provide  real-time  feedback;  highlights  aren’t  enough,  but  notes  might 
be. 

Workflow  Needs 

In  general,  users  want  many  workflow  tools  that  are  currently  not  being  designed: 
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•  “I  would  like  to  see  a  feature  where  I  can  approve  or  deny  additions/edits.  That’s 
how  it  is  in  AFATDS,  through  recommendations.  For  us,  wrong  information 
causes  a  lot  of  problems.” 

•  “(The  user)  should  be  able  to  set  this  as  a  setting  and  get  notifications.  To  get  an 
acknowledgement  once  someone  opens  the  board  (like  an  email  read  receipt).” 

•  “If  all  can  (see  it),  then  everybody  adds  together.  So  maybe,  check-out...  like 
Sharepoint.  A  Shared  drive  with  Sharepoint.  Check  the  map  in  and  out  like  a 
shared  folder.  Like  a  library.” 

•  “At  some  point  if  you  were  able  to  take  the  draft  and  the  certified  person  could 
approve  it...  it  could  be  a  workflow.” 

•  “No  watchers  ‘til  I  publish  it.  Closed  environment.  (This)  avoids  people  getting 
wrong  information  prior  to  publish.” 

•  (User)  wants  to  be  able  to  send  products  to  a  commander/etc...  for  a  review  before 
the  final  “publishing”:  “If  no  one  can  see  it,  just  to  make  damn  sure  it’s  right  before 
you  publish  to  the  world.” 

•  “I  talk  to  a  box,  not  a  person  so  (I  have)  no  real  problems  with  shift  changes.” 

•  “If  there  was  a  way,  once  someone  put  their  fingerprint  on  it  to  change  or  view  it, 
to  see  who  and  when,  alerting  the  person  who  runs  the  product,  it  would  be  good. 
Not  only  who  but  why.” 

•  “Show  a  screenshot  of  before  and  after  of  what  the  change  is.  Be  able  to  see  who 
made  that  on  his  role  and  battalion.” 


Next  Steps 

The  TacApps  design  team  will  use  the  insights  and  feedback  gathered  from  the  design 
workshop  to  continue  the  TacApps  design  using  understandable  and  familiar  patterns.  Affirmations 
of  existing  designs  will  help  harden  those  concepts  that  made  sense  to  participants,  and  places 
where  the  design  concepts  failed,  to  gain  broad  consensus  will  be  re-examined.  Also,  the  general 
knowledge  of  the  Soldiers’  workflows,  pain  points,  and  requirements  will  help  to  organize  upcoming 
design  work  by  user  priority. 

The  TacApps  design  team  is  planning  to  revisit  Fort  Riley  in  the  New  Year,  in  order  to  follow¬ 
up  on  the  Soldiers’  design  guidance  with  functional  software. 

In  addition  to  cross  walking  the  collected  user  feedback  against  existing  design 
documentation,  the  TacApps  design  team  will  also  compile  a  document  of  use  cases  that  were 
described  by  participants.  These  use  cases  will  be  helpful  tests  of  the  overall  TacApps  design, 
ensuring  that  there  is  system-wide  or  app-based  coverage  for  the  actions  and  requirements 
described  by  the  Soldiers. 

Another  document  that  can  be  generated  from  the  analysis  of  the  artifacts  this  session 
identified  is  a  list  of  future  application  ideas  for  TacApps.  The  app  ideas  from  participants  are 
included  briefly  in  figure  30  but  will  be  investigated  and  described  further  in  a  separate  document. 
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•  Spot  report  app 

•  Event  creator  app 

•  Review  and  comment/track  changes  workflow  app 

•  Task  org  manager 

•  Briefing  app 

•  Running  estimate  app  that  feeds  a  Commander’s  situational  awareness  dashboard 

•  Shift  change  organizer  app 

•  Automatic  S2  ordering  app 

•  Ordering  app  for  cargo,  supplies,  etc. 

•  Route  planning  and  navigation  app  that  lets  them  follow  the  ‘blue  line’  just  like  a 
phone’s  global  positioning  system 


Figure  30 

Participants’  app  ideas 


CONCLUSIONS:  EVENT  LESSONS  LEARNED 

•  The  Tactical  Applications  (TacApps)  design  team  is  planning  to  revisit  Fort  Riley,  KS,  in 
the  New  Year,  in  order  to  follow-up  on  the  Soldiers’  design  guidance  with  functional 
software. 

•  The  paper  prototype  navigation  didn’t  work.  It  was  too  unwieldy  and  required  too  much 
preparation  in  advance  by  guides  who  could  only  possibly  do  one  at  a  time.  It  was  hard 
to  document  after  the  fact.  If  this  is  ever  done,  it  should  be  documented  electronically  so 
that  the  navigation  path  is  recorded  automatically. 

•  Facilitating  the  paper  prototype  activity  seemed  awkward,  and  it  was  difficult  to  extract  the 
participant’s  thought  process  (products,  activity  2,  and  handout  3-3). 

•  It  was  difficult  sustaining  a  dialogue  with  the  participants  during  this  session,  which  was 
much  too  long,  based  on  their  limited  feedback  (products,  activity  4). 

•  As  with  the  previous  activity,  it  was  difficult  to  engage  with  the  participants,  hence  the  light 
amount  of  feedback  (products,  activity  5). 

•  Scribes  should  be  given  a  standard  paper  type  to  write  on  -  one  that  is  easy  to 
automatically  scan  after  the  fact. 

•  The  people  images  on  the  handout  folder  icons  were  too  light/washed  out  to  be  seen.  An 
indelible  marker  or  ballpoint  pen  should  be  used  to  make  drawings  and  other  markup 
easier  to  see  on  a  scanned  copy. 
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Survey 

•  Years  of  Mission  Command  experience  question  is  vague. 

•  “Command  experience?  Is  that  DIV?  I’ve  got  20  years,  I’ll  say  that.”  -  Participant. 

•  “Background  wasn’t  clear.  Needs  to  be  crystal  clear  for  the  military;  be  specific. 
Participant. 

•  Users  didn’t  always  know  the  survey  was  two  sided. 

•  Did  not  include  question  that  specifically  asked  for  rank,  forcing  scribes  to  record  this  data 
separately. 

•  “Number  of  years  you  expect  to  serve”  should  have  been  requested  to  separate  the 
practically-concerned  from  the  theoretically  concerned. 

Presentation 

•  The  onboarding  slides  were  missing. 

•  Some  participants  weren’t  totally  clear  on  “why”  they  were  watching  the  movies. 

•  'Am  I  judging  what  that  video  explained?”  -  Participant. 

•  In  activity  3,  confusion  was  created  by  calling  the  collaborative  example  a  battle  update 
brief  (BUB).  In  reality,  a  participant  explained,  a  BUB  has  a  formal  update  cycle,  and  is 
otherwise  static,  so  the  BUB  work-product  doesn’t  fit  with  the  workflow  described  in  that 
example.  The  participant  recommended  that  it  should  have  been  called  the  concept  of 
operations  worksheet,  which  has  a  more  dynamic  updating  rhythm. 

•  Some  of  the  Soldiers  were  looking  disinterested  by  the  end  of  the  introduction  -  maybe 
shorten  it  for  next  time  or  include  visual  aids. 
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APPENDIX 
BACKUP  DATA 
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Participant  Quotes 

•  "Yes,  I’m  excited  about  this  s**t.  I  can’t  wait  to  talk  to  you  guys."  -[User],  during  the 
introduction. 

•  "There's  an  app  that  can  do  that  in  a  second.  This  is  2015!"  --regarding  a  cumbersome  widget 
ordering  process  (filling  out  forms,  Excel  spreadsheets,  tracking,  statusing,  fund  allocation, 
etc.). 

•  "I  just  like  the  'usual'. " 

•  "We  like  s**t  to  look  cool. discussing  the  perspective  of  millennials 

•  "Everything  I  learned  about  CPOF  was  through  trial  and  error. " 

•  "You  want  me  to  just  build  this  for  you  guys?  When  I  heard  there  was  a  [Design  Workshop]  for 
the  replacement  for  CPOF,  I  was  like:  'Get  me  in  on  that  right  now!'  as  I  closed  my  CPOF 
terminal. " 

•  "I  didn 't  know  this  much  thought  went  into  creating  the  software. " 

•  "It's  2015!"  —  objecting  to  4-week  training  requirement  for  DCGS-A. 

•  "I  think  you  should  take  the  Apple®  approach  and  make  it  super  simple  so  you  can't  fail.  It's  all 
about  keeping  it  simple. " 

•  "We'll  be  modern  combat  fighters  eventually. "  —  discussing  bandwidth,  satellites,  etc. 

•  "Collaboration  causes  issues." 

•  "CPOF  is  like  Pangaea.  Good,  but  a  million  years  ago.  It’s  time  to  let  all  that  stuff  separate 
out." 

•  "Screen  grabs  on  a  paste  board  look  like  a  2 year-old  did  it. " 

•  "Fm  scared  as  hell  of  that  thing,  I  don 't  want  tof***  with  that!"  (re:  CPOF) 

•  "Office  products  don’t  allow  you  to  do  what  CPOF  can  like  change  and  add  a  product.  In  Office, 
you  copy  or  check  in.  CPOF  is  pretty  cool  for  that.  ” 

•  "Why’s  it  always  the  Major?!"  (remarking  on  the  fairness  of  Major  Smith  always  owning  all 
the  products  shown  in  the  videos). 

•  “Major  Smith  is  paranoid. "  (re:  our  video  examples) 

o  "Bob  and  joe  are  collaborating.  Major  Smith  is  the  feudal  lord. "  (re:  our  video 
examples). 

o  “In  the  fog  of  war,  you  could  plan  the  perfect  plan  but  it  won ’t  last.  To  be  able  to 
actively  push  out  'hey  I  updated  this’  would  be  great." 

•  “Mirror,  clone,  replicate.  [Functionality]  similar  to  that  would  be  ideal." 

•  "If  I  need  their  air  corridor,  I’d  mirror  it,  so  any  changes  they  make  I’ll  get.  If  it’s  an 
operational  graphic,  sometimes  taking  it  and  completely  breaking  the  link  can  be  helpful." 

•  7  hate  engineers;  they  design  cars  but  don’t  maintain  [them]." 

•  "Today’s  army  makes  things  a  little  more  complicated  than  they  actually  are. " 

•  "You  could  create  a  tutorial  for  each  topic,  if  you  want  to  do  the  walkthrough  tutorial.  I’ve 
seen  this  for  games,  like  you’ve  achieved  certain  things,  [you]  get  this  many  points." 

•  7  can  tell  the  people  who  just  watched  the  IMI  versus  those  who  have  taken  the  course. " 
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•  "[It  takes]  a  reasonable  amount  of  time — maybe  a  week  long  class — to  learn  a  system  like 
this." 

•  "The  army’s  using  a  training  matrix/table  system  now.  It  includes  the  basics  of  the  job  I’m 
doing  now,  and  has  lots  of  smaller  pieces  that  you  can  do  one  at  a  time,  and  it  tracks  your 
progress  as  you  complete  modules." 

•  “Everything  is  competitive  in  the  army." 

•  "If you  guys  could  just  make  [TacApps]  work  on  a  laptop,  that'd  be  great  [instead  of  having  to 
go  to  a  specific  computer]." 

•  "Civilization,  the  game,  starts  off  with  a  tutorial  but  it  also  has  a  help  menu...  very  visual.  Have 
an  option  to  turn  on  or  off  the  tutorial." 

•  "Whenyou  get  a  new  phone,  it  has  a  quick  setup  wizard.  That  works  well." 

•  7  love  Clippy!"  [User,  re:  Microsoft®  Clippy,  sarcasm  uncertain]. 

•  7  don 't  like  the  computer  talking  but  [I  like]  if  I’m  in  a  library  and  I  can  see  the  librarian  open 
a  chat  session  and  help  me." 

•  "Somehow  I  want  to  know  what  everyone  else  is  doing. " 

•  "I  hate  chat  bot.  I  don 't  want  somebody  chatting  me. " 

•  “Phone  games  work  well.  Clash  of  Clans.  It  tells  you  'yo — before  you  do  anything,  build  a  town 
hall’  etc...”  (re:  onboarding). 

•  "CPOF  cannot  command  and  control.  I  cannot  control  my  [stuff]  through  CPOF.  It’s  a  hell  of  a 
briefing  tool  though." 

•  “Warfighter  function  changes  more  frequently  than  staff  function." 

•  “I'm  an  infantryman.  It’s  bad  news  if  I  start  making  up  my  own  call  sign." 

•  "Unicorn  slayer.  That's  what  somebody's  gonna  put"  (re:  The  concept  of  self-designated  skills 
being  part  of  user  identity). 

•  "Special  Skills:  dangerous,  people  put  all  kinds  of  stuff.  If  a  platoon  leader  needs  a  cement 
mixer,  he  is  going  to  ask  company  if  there  is  an  engineering  asset  to  task  a  cement  mixer  to  the 
operation. " 

•  “Staff function  would  be  the  shop  you’re  in.  Role  is  like  S3  or  CUOP." 

•  “Warfighter  function  in  CPOF  is  what  their  job  is,  like  map  builder. " 

•  "Oh,  I  want  to  be  able  to  type.  There  better  be  somewhere  for  me  to  type.  Give  me  a  place  to 
type!"  -  (re:  Wanting  direct  text  input  in  addition  to  dropdown  criteria  by  unit). 

•  "I  understand  what  they  were  trying  to  do  with  CPOF  in  the  lab,  maybe.  It's  so  unusable. " 

•  "[It  would]  be  cool  to  have  favorite  users.  Click  on  them  and  boom — see  their  stuff  or  talk." 

•  "An  auto-populated  most  useful  user  list  based  upon  the  number  of  times  I  clicked  on  them. " 

•  [User’s]  examples  of  a  hypothetical  individual’s  self-assigned  roles:  "In-flight  missile  repair’ 
and  'Space  shuttle  door  gunner." 

•  "I’m  partial  to  horizontal  scrolling." 

•  "We've  seen  guys  taking  their  shirts  off,  ready  to  go  at  it  because  somebody  jacked  up  their 
entire  presentation  right  before  a  brief" 

•  “Most  system  users  understand  nothing's  ever  done — war  moves  so  fast." 
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•  "My  experience  with  Army  systems  is  that  TacApps  will  take  five  years  to  go  Army -wide. " 

•  [Two  users]  mention  that  they  will  be  out  of  the  military  by  the  time  this  system  likely 
deploys. 

•  "Sometimes  it's  best  not  to  allow  these  things  to  happen. " 

•  " Permissions  really  suck. " 

•  "This  is  going  to  be  great!" 

•  "You  could  build  a  gun  that  shoots  colored  gerbils  against  white  walls  and  it  would  be  better 
than  CPOF. " 

•  "Users  do  not  want  to  be  injected  with  smart  chips  in  their  necks.  What  are  we  now,  puppies?" 

•  "Keep  it  as  close  to  what  we  work  with.  Windows®. " 

•  "Two  windows  [side-by-side]  —  I'd  never  have  a  need  for  that. " 

•  "I’m  a  dismount  so  I’m  not  a  computer  guy. " 

•  "The  COP  is  the  most  important  thing. " 

•  7  want  to  deal  with  something  that  produces  a  record,  so  it’s  not  just  my  word  but  there’s  a 
record  of  it." 

•  "CPOF  is  tough  to  understand  because  it’s  not  like  Windows®.  But  once  you  learn  it,  it  can  be 
OK." 

•  “Everyone  in  the  military  is  familiar  with  Microsoft®. " 

•  "Permissions  are  powerful  but  people  don 't  understand  them. " 

•  "Embedding  the  help  would  definitely  help. " 

•  "TIGR  sets  a  really  good  example  for  an  easy-to-use  system.  I  like  being  able  to  use  the  search 
tool  by  MGRS  and  that  it  highlights  items/efforts/work  products  in  that  area. " 

•  "Lots  of  Intel  analysts  use  Google  Earth®. " 

•  "Backup!  You  know,  it  might  seem  like  a  really  great  idea,  but  it  never  happens. "  (re:  having 
an  SOP-assigned  backup  for  warfighter  functions). 

•  "CPOF  would  not  work  well  on  a  tablet.  Putting  a  map  and  efforts  together  and  trying  to  edit 
them  would  not  work." 

•  "The  Army  has  a  lot  of  different  reports.  A  simple  'Create  Event’  button  would  be  good.  There 
are  hundreds  of  doctrinal  icons  that  nobody  uses." 

•  "[In]  CPOF  right  now,  it  takes  so  much  time  to  enter  things,  like  a  SPOT  report." 

•  "Here’s  the  big  problem;  all  the  systems  are  supposed  to  communicate  but  in  reality  they  don 't. 
So  we  recreate  stuff” 

•  “We're  trying  to  move  away  from  PowerPoint.  The  only  reason  we  use  PowerPoint  is  that  all 
the  systems  can  take  them  in.  There’s  no  other  functionality  or  preference  reason.” 

•  "[User]  likes  the  Ventrilo  style  online  presence  indicator  of  who’s  actively  attending  the  brief" 

•  "Sharing  the  map  is  problematic  because  so  many  people  would  mess  with  it.  [We’ve]  had  to 
move  it  to  Brigade  level.  Causes  bad  tension." 

•  "If you  're  not  trained  to  do  something  [CPOF],  you  won 't  figure  it  out. " 

•  "The  Trash  bin  is  essential.  I  would  like  anything  IN  the  trash  to  stop  running.  To  get  no 
updates  and  run  no  automation." 
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•  "Task  Org  shouldn 't  change  much,  unless  a  lot  of  people  die  and  they  start  combing  battalions. " 


Participants’  Feature  Requests 

•  Spellcheck  (so  they  don’t  have  to  use  MS  Word) 

•  Ability  to  merge  products/layers  together 

•  Ability  to  templatize  a  product  and  modify  as  needed 

•  Search,  with  advanced  filters,  across  the  entire  system  of  apps 

•  A  training  database,  so  that  learners  can’t  impact  real  products 

•  Configurable  ‘hot  buttons’  (hot  keys) 

•  Self-paced  User  Guide  within  the  system 

•  Ability  to  move  the  taskbar  to  anywhere  it’s  convenient 

•  Tagging 

•  A  universal,  or  at  least  group,  login  for  convenience,  followed  by  individual 
CAC  login 

•  A  banner  on  products:  who  the  user  is  sharing  with,  or  ‘nobody’  for 
transparency 

•  Hover-over  on  folders  that  displays  official  group  contact  info,  like  telephone 
numbers/email 

•  A  database  of  templates 

•  Ability  to  send  a  Fragmentary  Order  (FRAGO)  across  many  groups  as  an 
urgent  notification 

•  Ability  to  import/export  Google  Earth  KML  files 

•  Enemy  position  that  shows  up  on  AFATDS  and  TacApps  simultaneously 

•  Do  NOT  use  spacebar  to  pan  the  map 

•  Combine  BCS3  and  [TacApps]  functionality  somehow 

•  A  last  modified  who/when  on  all  products 

•  Share/  import/export  graphics 

•  A  visual  ‘dif;  shows  all  changes  on  the  map  with  different  colors.  Color- 
coded  by  person  or  by  the  time  since  the  change...  red  for  recent  changes 
and  green  for  the  ‘previous  24  hours’ 
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New  and  Interesting  Terminology  from  Participants 

•  Common  (as  in  a  shared  map) 

•  Check  in  and  check  out  (version  control) 

•  Send  up /shoot  up  (to  a  higher  echelon) 

•  Blast  ( send  wide,  quickly  and  for  crucial  notifications) 

•  MFWS  (multi-function  workstation) 

•  !/Ka//(for  public  sharing,  like  Facebook) 

•  Squatting  (placing  data  in  a  unit’s  SOP-approved  ‘edit  status’  for  too  long) 

•  Drive  (to  control  the  briefing) 

•  Permissions  (participants  near-universally  used  this  term  rather  than 
privileges) 

•  Merge  (combine  multiple  products-or  their  layers-into  one) 

•  Living  document  (a  product  with  live  data) 

•  Feed  (as  in  ‘to  feed  Battalion  with  information’) 

•  Dialog  ( to  talk) 

•  Stalk  ( to  watch  for  changes  on  a  piece  of  data  or  a  product) 

•  instance  (a  single  version  of  a  visualization) 

•  Observe  or  participate  (briefing  roles) 

•  Everyday  terms  (common  language) 

•  Orient  (to  learn  a  map  or  visualization) 

•  Bleed  out  (how  info  spreads  through  a  network  of  people  or  products) 

•  Shared  drive 

•  EDRE (Emergency  Development  Response  Exercise) 

•  FEC(F\'ce  Engagement  Cell) 

•  Pull  vs.  push  data 


Mention  of  C.O.T.S.  Tools  by  Participants 

Throughout  the  Design  Workshop  sessions,  as  participants  described  their  ideal  functionality 
(or  even  their  most-disliked  functionality)  the  design  team  scribes  took  note  of  the  various 
commercial,  off-the-shelf  software  tools  that  they  used  to  illustrate  their  points.  These 
mentions,  tabulated  below  in  a  word  cloud  (in  which  the  largest  font  indicates  the  highest 
number  of  mentions;  15  for  PowerPoint),  reveal  the  participants’  familiarity  with  external 
systems.  Taken  in  aggregate,  this  can  provide  insight  into  the  design  patterns  that  could 
provide  the  most  broadly  successful  recognition  if  refused  in  TacApps. 
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Popularity  of  Various  COTS  Tools  as  Described  by  the  Participants  in  Their  Own  Words 
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DISTRIBUTION  LIST 


U.S.  ArmyARDEC 
ATTN:  RDAR-EIK 

RDAR-WSF-M,  A.  Lieb 
R.  Arnold 

Picatinny  Arsenal,  NJ  07806-5000 

Defense  Technical  Information  Center  (DTIC) 
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REVIEW  AND  APPROVAL  OF  ARDEC  REPORTS 

THIS  IS  A: 

3  TECHNICAL  REPORT  □  SPECIAL  REPORT 


□  MEMORANDUM  REPORT 

□  ARMAMENT  GRADUATE  SCHOOL  REPORT 

FUNDING  SOURCE  PM  funded  EMD 

[e  g..  TEX3:  6.1  (ILIR.  FT  AS);  6  2;  6.3; 

TacApps  User  Design  Workshop 

PM  funded  EMD:  PM  funded  Producliorii'ESIP;  Olrier  (please  ideniity)] 

Tactical  Applications 

Title 

Ross  Arnold 

Project 

A'uthor/Prcject  Engineer 

x3618  S9 

Report  numbertCaie  received  (to  be  completed  by  LCSD) 

RDAR-W5F-M 

Extension  Building 

Author's  Office  Symbol 

PART  1.  Musi  be  signed  before  the  report  can  be  edited. 

a.  The  draft  copy  of  this  report  has  been  reviewed  lor  technical  accuracy  and  is  approved 
foradlfng. 

b.  Use  Distribution  Statement  A  x  .  B _ ,  C _ ,  D _ ,  E _ ,  or  F _ for  »>e  reason 

checked  on  the  oonilrwalton  ol  this  form.  Reason: _ 

1.  If  Statement  A  Is  selected,  the  report  trill  be  released  to  the  National  Technical 
information  Service  (NTIS)  for  sate  to  the  general  public.  Only  unclassified  reports 
whose  distribution  is  not  limited  or  controlled  In  anyway  are  released  to  NTIS. 

2.  It  Statement  B,  C,  D.  E,  or  F  is  selected,  the  report  will  b<s  relcassd  to  the  Defense 
T echnicaf  Information  Center  (DTIC)  which  will  limit  distribution  according  to  the 
conditions  indicated  in  lire  statement. 

o.  The  distribution  list  for  this  report  has  been  reviewed  lor  accuracy  and  completeness. 

Tri  Lu 

"Di^SorTchteT 


PART  2.  To  be  signed  either  when  draft  report  is  submitted  or  after  review  of  reproduction  copy. 


This  report  is  approved  for  publcation. 


LCSD  49  (1  Sept  16) 

Supersedes  SMCAR  Form  49,  20  Dec  06 


Tri  Lu 


Bivisjnrt  Chic? 


xmLaS'Y' 

/  ‘  (Date) 


Andrew  Pskowski 


ROAR-CIS 
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